Method for providing location independent dynamic port mirroring on distributed virtual switches

ABSTRACT

Techniques for providing location independent dynamic port mirroring on distributed virtual switches is disclosed. A controller is provided to configure one or more virtual switches within a group of physical machines to appear as a set of distributed virtual switches. In response to the receipt of a data packet at a port of a physical machine, a determination is made whether or not the port has a monitor port located on the physical machine. If the port has a monitor port located on the same physical machine, a copy of the data packet is sent to the monitor port of the physical machine. If the port has a monitor port located on a different physical machine, a copy of the data packet along with an identification (ID) of the port and an ID of the monitor port are encapsulated, and the encapsulated information are sent to a controller.

BACKGROUND OF THE INVENTION

1. Technical Field

The present disclosure relates to server virtualizations in general, andin particular to distributed virtual switches. More particularly, thepresent disclosure relates to a method for providing locationindependent dynamic port mirroring on distributed virtual switches.

2. Description of Related Art

Generally speaking, server virtualization describes a softwareabstraction that separates a physical resource and its use from theunderlying physical machine. Most physical resources, such asprocessors, storage devices, and network adaptors, can be abstracted andprovisioned as virtualized entities.

Virtual machines (VMs) play a central role in server virtualization. AVM is a virtualization of a physical machine and its hardwarecomponents. A VM typically includes a virtual processor, a virtualsystem memory, and various virtual devices. A single physical machinecan host multiple VMs. Guest operating systems can be executed on VMsand function as though executing on actual hardware of a physicalmachine.

A hypervisor or virtual machine manager provides an interface betweenVMs and the underlying hardware of a physical machine. By multiplexingall accesses to the underlying hardware among various VMs, a hypervisorguarantees various VM the usage of the actual hardware, such asprocessors, system memory, etc., of the physical machine.

A typical server virtualization implementation generally requiresmultiple VMs to share a network adapter or network interface card (NIC)of a physical machine for performing external network input/outputoperations. A hypervisor typically provides a virtual switch (vswitch)that provides interconnectivity among the VMs on the physical machine.With each VM having one or more virtual NICs (vNICs), the vswitchinterfaces between the NIC of the physical machine and the vNICs of theassociated VMs. In general, each vNIC operates like a physical NIC,being assigned a media access control (MAC) address that is typicallydifferent from that of the physical NIC. The vswitch performs therouting of packets between the various vNICs and the physical NIC.

The present disclosure provides an improved method for providing portmirroring on distributed vswitches.

SUMMARY OF THE INVENTION

In accordance with a preferred embodiment of the present disclosure, acontroller is provided to configure one or more virtual switches withina group of physical machines to appear as a set of distributed virtualswitches. In response to the receipt of a data packet at a port of aphysical machine, a determination is made whether or not the port has amonitor port located on the physical machine. If the port has a monitorport located on the same physical machine, a copy of the data packet issent to the monitor port of the physical machine. If the port has amonitor port located on a different physical machine, a copy of the datapacket along with an identification (ID) of the port and an ID of themonitor port are encapsulated, and the encapsulated information are sentto a controller.

All features and advantages of the present disclosure will becomeapparent in the following detailed written description.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure itself, as well as a preferred mode of use, furtherobjects, and advantages thereof, will best be understood by reference tothe following detailed description of an illustrative embodiment whenread in conjunction with the accompanying drawings, wherein:

FIG. 1 is a block diagram of a data center having multiple physicalmachines in which a preferred embodiment of the present invention can beimplemented;

FIG. 2 is a block diagram of a logical representation of the data centerfrom FIG. 1;

FIG. 3 is a diagram illustrating the relationships between end-nodes anddownlink virtual ports from FIG. 2;

FIG. 4 is a block diagram of an overall architecture of locationindependent dynamic port mirroring on distributed virtual switches, inaccordance with a preferred embodiment of the present invention; and

FIG. 5 is a high-level logic flow diagram of a method for providinglocation independent dynamic port mirroring on distributed virtualswitches, in accordance with a preferred embodiment of the presentinvention.

DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT

Referring now to the drawings and in particular to FIG. 1, there isdepicted a block diagram of a data center having multiple physicalmachines in which a preferred embodiment of the present invention can beimplemented. As shown, a data center 10 includes multiple physicalmachines 12 a-12 n in communication with a network 14 through a networkswitch 19. Network 14 can be a local-area network (LAN), a wide-areanetwork (WAN), or a metropolitan-area network (MAN). The equipment ofdata center 10 can reside together locally at a single site ordistributed over separate sites.

Each of physical machines 12 a-12 n may include hardware such asprocessors, memories, input/output (I/O) devices, network interfacecards (NICs) or converged network adapters (CNAs), etc. Physicalmachines 12 a-12 n can reside alone or be stacked together within achassis such as in a rack server or in a blade server, and networkswitch 19 can reside alone or be stacked within the same chassis as oneor more of physical machines 12 a-12 n.

Each of physical machines 12 a-12 n may host one or more end-nodes. InFIG. 1, physical machine 12 a is shown to host two end-nodes 18 a and 18b, and physical machine 12 n is shown to host one end-node 18 n.End-nodes 18 a-18 n can be physical or virtual. Examples of end-nodes 18a-18 b include, but are not limited to, application programs, operatingsystems, virtual machines, hypervisors, virtual and physical NICs,virtual and physical NIC queues, and virtual and physical storagedevices. Types of end-nodes 18 a-18 b include, but are not limited to,network end-nodes and storage end-nodes. Network end-nodes processnetwork data packets, and storage end-nodes process storage datapackets. Physical and virtual end-nodes that perform data networking arecalled physical and virtual network end-nodes, respectively, whereasphysical and virtual end-nodes that perform storage networking arecalled physical and virtual storage end-nodes, respectively.

Network switch 19 includes multiple physical downlink ports 15 a-15 nand multiple physical uplink ports 16 a-16 n. Network switch 19 performsswitching of data packets between physical downlink ports 15 a-15 n andphysical uplink ports 16 a-16 n. Each of physical machines 12 a-12 n isdirectly connected to one of physical downlink ports 15 a-15 n via acorresponding one of physical links 13 a-13 n. Physical uplink ports 16a-16 n serve to connect network switch 19 to network 14 via physicaluplinks 17 a-17 n.

Network switch 19 may include a management module 11 by which networkswitch 19 is configured to perform switching of data packets based onvirtual ports (v-ports). An Ethernet switch is an exemplaryimplementation of network switch 19.

With reference now FIG. 2, there is depicted a logical representation ofdata center 10. As shown, data center 10′ includes end-nodes 28 a-28 nin communication with a v-port switch 29. V-port switch 29 is a networkelement that can learn of the existence and identities of one or moreend-nodes of a physical machine, and can detect, monitors, and controlsdata packet traffic to and from those end-nodes. In FIG. 2, each ofend-nodes 28 a-28 n is logically connected to a different one of virtualports (v-ports) 25 a-25 n of v-port switch 29. Each of v-ports 25 a-25 nis uniquely assigned to one of end-nodes 28 a-28 n. For example, v-port25 a is logically connected to end-node 28 a via virtual downlink 23 a;v-port 25 b to end-node 28 b via virtual downlink 23 b; and v-port 25 nto end-node 28 n via virtual downlink 23 n.

End-nodes 28 a-28 n are computing or traffic-handling entities operatingon physical machines 12 a-12 n connected to a physical port of v-portswitch 19. Such entities can be physical entities, such as a networkinterface card (NIC), or virtual entities, such as a virtual NIC of avirtual machine.

The generation of a v-port for v-port switch 29 can occur staticallythrough administrator configurations or dynamically (i.e., in real-time)through end-node discovery and automatic v-port assignments. V-portswitch 29 uses v-ports 25 a-25 n in similar fashions to those ofphysical ports. Because full physical port functionality can be extendedto v-ports 25 a-25 n, each one of v-ports 25 a-25 n is treated as havingat least the same capabilities as a physical port.

The associations of v-ports 25 a-25 n to end-nodes 23 a-23 n areone-to-one. Examples of end-node associations of v-ports include, butare not limited to, an association with a virtual NIC or a subsetthereof of a virtual machine operating on a physical machine,associations with different queues of a multi-queue NIC or a subsetthereof on a physical machine, associations with different networkqueues or a subset thereof of a CNA, and associations with differenttypes of traffic on a CNA, such as Fibre Channel over Ethernet (FCoE)traffic.

V-port switch 29 also defines uplink v-ports 26 a-26 n that arelogically connected to physical uplink ports 16 a-16 n (from FIG. 1) byvirtual uplinks 27 a-27 n. Each of virtual uplinks 27 a-27 n has aone-to-one correspondence with one of uplink v-ports 26 a-26 n, andconnects that uplink v-port to one of physical uplink ports 16 a-16 n(from FIG. 1). Multiple virtual uplinks 27 a-27 n, and thus multipleuplink v-ports 26 a-26 n, can logically connect to the same physicaluplink port 16 a-16 n. Each one of downlink v-ports 25 a-25 n islogically associated with one of uplink v-ports 26 a-26 n, with morethan one of downlink v-ports 25 a-25 n possibly being associated withany given one of uplink v-port 26 a-26 n. When a data packet arrives atv-port switch 29 via one of downlink v-ports 25 a-25 n, v-port switch 29switches the data packet to the associated one of uplink v-ports 26 a-26n, and from the uplink v-port, switches the data packet to theparticular one of physical uplink port 16 a-16 n to which the uplinkv-port is logically connected.

Referring now to FIG. 3, there is illustrated the relationships betweenend-nodes and downlink virtual ports from FIG. 2. As shown, multipleend-nodes operate within virtual machines (VMs) connected to the samephysical interface. Physical machine 12 a (from FIG. 1) hasvirtualization software that includes hypervisor 30 for abstracting thehardware of physical machine 12 a into one or more VMs 31 a, 31 b and 31c.

Each one of VMs 31 a-31 c has one or more associated virtual interfaces(VIF), such as a virtual NIC, with each VIF having its own uniquevirtual MAC address (vMAC). In FIG. 3, virtual machines 31 a, 31 b bothhave one VIF 34 a, 34 b, respectively, and virtual machine 31 c has twoVIFs 34 c, 34 d. In addition, each one of VMs 31 a-31 c includes atleast one software application executing within its own guest operatingsystem. Any type of application can execute on one of VMs 31 a-31 c.

Each one of VIFs 34 a-34 d is an example of a virtual end-node. A givenone of VIFs 34 a-34 d can be configured to handle data networking orstorage communications. VIFs that process data networking communicationsare examples of virtual network end-nodes, and VIFs that process storagecommunications are examples of virtual storage end-nodes.

Hypervisor 30 is in communication with a NIC 60 that handles the I/Os toand from v-port switch 29. Through hypervisor 30, VIFs 34 a-34 d arelogically connected to NEC 60 via virtual links 38.

NIC 60 is connected to a physical port 32 a by a physical link 39 a.Logically associated with physical port 32 a, as signified by virtuallinks 36 a-36 d, are four downlink v-ports 25 a-25 d. Each one ofdownlink v-ports 25 a-25 d is uniquely assigned to one of virtualend-nodes VIF 34 a-34 d. For example, v-port 25 a can be assigned to VIF34 a; v-port 25 b to VIF 34 b; v-port 25 c to VIF 34 c; and v-port 25 dto VIF 34 d. These four downlink v-ports 25 a-25 d can also beconsidered logically associated with physical link 39 a; that is, eachone of downlink v-ports 25 a-25 d is a subdivided part of physical link39 a.

With reference now to FIG. 4, there is illustrated a block diagram of anoverall architecture of location independent dynamic port mirroring ondistributed virtual switches, in accordance with a preferred embodimentof the present invention. In order to implement the port mirroringfeature where a source port and a monitor port are located on separatephysical machines, the solution needs to include other components thatare part of the distributed virtual switch (DVS) solution. For example,a central controller 41 is needed for a network administrator toconfigure the DVS and port settings. There is a one-to-one mappingbetween a DVS and a controller. In addition, kernel-mode modules 42 a-42n are included in corresponding physical machines 12 a-12 n for handlingpacket forwarding. Also, user-mode modules 43 a-43 n can be optionallyincluded in corresponding physical machines 12 a-12 n for handling somecontrol plane protocols.

Each one of kernel-mode modules 42 a-42 n and each one of user-modemodule 43 a-43 n interact with one another using operation systemspecific mechanisms. On the other hand, each one of user-mode modules 43a-43 n communicates with controller 41 via sockets. The presentinvention utilizes the communication paths among kernel-mode modules 42a-42 n, user-mode modules 43 a-43 n and controller 41 to encapsulate andto send data packets from a kernel-mode module on one of physicalmachines 12 a-12 n to a kernel-mode module on a different one ofphysical machines 12 a-12 n.

The multiplexing of the (mirrored) data packet destination is performedby controller 41. To accomplish this, controller 41 must be able to knowthe (socket) location of the user-mode module at each physical machine,and the user-mode module at the source port (end of the port mirror)must encapsulate sufficient information in the encapsulated data packetsso that controller 41 can decide which one of physical machines 12 a-12n the encapsulated packet should be directed to.

In order for a VM to come alive on a v-port, the v-port needs to reportto controller 41 initially. For example, in FIG. 3, VIF 34 a isconnected to hypervisor 30 on physical machine 12 a, and VM 31 a becomesalive on physical machine 12 a after VIF 34 a sends a message to acontroller (not shown). In turn, the controller records the location ofVIF 34 a in its database. A VIF can be alive on only one physicalmachine at any given time. If the VIF moves to a different physicalmachine during migration, the VIF will have to disconnect from thecurrent physical machine before coming alive on the different physicalmachine. For example, after VIF 34 a has been disconnected fromhypervisor 30 on physical machine 12 a, VIF 34 a sends a message to thecontroller, and VM 31 a is not alive on physical machine 12 a anymore.The controller then removes the location of VIF 34 a from its database.

A user can assign and configures a monitor port of a (mirrored) port ona distributed switch by using a management tool. For example, VIF 34 acan have a monitor port VIF 34 a′.

Referring now to FIG. 5, there is depicted a high-level logic flowdiagram of a method for handling local port mirroring on distributedvirtual switches, in accordance with a preferred embodiment of thepresent invention. Starting at block 50, in response to an incoming datapacket arriving at a port (e.g., VIF 34 a) of a physical machine (e.g.,physical machine 12 a), a determination is made whether or not the porthas a monitor port, as shown in block 51.

If the port has a monitor port (e.g., monitor port VIF 34 a′), anotherdetermination is made whether or not the monitor port is located on thesame physical machine (e.g., physical machine 12 a), as depicted inblock 52. If the monitor port is not located on the same physicalmachine, then a copy of the data packet, the source port ID (i.e., portID of VIF 34 a) and the monitor port ID (i.e., port ID of VIF 34 a′) areencapsulated and sent to the controller (e.g., controller 41), as shownin block 53. The controller then determines whether or not the monitorport ID is stored in its database, as depicted in block 54. If themonitor port ID is stored in the controller database, then thecontroller sends the data packet to the location of the monitor port IDbased on the information stored in the controller database, as shown inblock 55. Otherwise, if the monitor port ID is not stored in thecontroller database, the data packet is dropped, as depicted in block56.

Otherwise, if the monitor port is located on the same physical machine,then a full copy of the incoming data packet is sent to the monitor port(i.e., monitor port VIF 34 a′), as shown in block 57.

As has been described, the present disclosure provides a method forproviding location independent dynamic port mirroring on distributedvirtual switches.

It is also important to note that although the present invention hasbeen described in the context of a fully functional computer system,those skilled in the art will appreciate that the mechanisms of thepresent invention are capable of being distributed as a program productin a variety of recordable type media such as compact discs and digitalvideo discs.

While the disclosure has been particularly shown and described withreference to a preferred embodiment, it will be understood by thoseskilled in the art that various changes in form and detail may be madetherein without departing from the spirit and scope of the disclosure.

1-4. (canceled)
 11. A computer readable medium having a computer programproduct for providing location independent dynamic port mirroring, saidcomputer readable medium comprising: program code for, providing acontroller for configuring one or more virtual switches within aplurality of physical machines to appear as a set of distributed virtualswitches; program code for, in response to the receipt of a data packetat a port of one of said physical machines, determining whether or notsaid port has a monitor port located on said one physical machine;program code for, in a determination that said port has a monitor portlocated on said one physical machine, sending a copy of said data packetto said monitor port; and program code for, in a determination that saidport has a monitor port located on a different one of said physicalmachines, encapsulating a copy of said data packet along with anidentification (ID) of said port and an ID of said monitor port, andsending said encapsulated information to said controller.
 6. Thecomputer readable medium of claim 5, wherein said computer readablemedium further includes program code for, determining by said controllerwhether or not said monitor port ID is stored in a database within saidcontroller; program code for, in a determination that said monitor portID is stored in a database within said controller, sending said datapacket to said monitor port according to said monitor port ID stored insaid database within said controller; and program code for, in adetermination that said monitor port ID is not stored in a databasewithin said controller, dropping said data packet.
 7. The computerreadable medium of claim 5, wherein said virtual switches are containedwithin a hypervisor of each of said physical machines.
 8. The computerreadable medium of claim 7, wherein said controller communicates witheach of said physical machines via a communication module within ahypervisor of each of said physical machines.